YORO - Lightweight End to End Visual Grounding

We present YORO - a multi-modal transformer encoder-only architecture for the Visual Grounding (VG) task. This task involves localizing, in an image, object referred via natural language. Unlike recent trend literature of using multi-stage approaches that sacrifice speed accuracy, seeks better trade-off between accuracy by embracing single-stage design, without CNN backbone. consumes language queries, image patches, and learnable detection tokens predicts coordinates object, single encoder. To assist alignment text visual objects, novel patch-text loss is proposed. Extensive experiments are conducted on 5 different datasets with ablations design choices. shown to support real-time inference outperform all this class (single-stage methods) large margins. It also fastest VG model achieves best speed/accuracy literature. Code released (Code available at https://github.com/chihhuiho/yoro ).